Skip to content

fix memory leak#14

Merged
shuai-xu merged 1 commit intobigo-sg:gluten-0530from
xinghuayu007:test_bigo
Oct 29, 2025
Merged

fix memory leak#14
shuai-xu merged 1 commit intobigo-sg:gluten-0530from
xinghuayu007:test_bigo

Conversation

@xinghuayu007
Copy link

Bug description.

std::shared_ptrio::IoStatistics ioStats passes it's naked pointer to FileSink::create(), then ioStats is released. Therefore FileSink holds an unknown pointer.

AddressSanitizer: heap-use-after-free on address 0x6130004f1998 at pc 0x7f6490a61210 bp 0x7f64aaed8e90 sp 0x7f64aaed8e88
[ERROR] WRITE of size 8 at 0x6130004f1998 thread T146 (Legacy Source T)

[ERROR] #0 0x7f6490a6120f in std::__atomic_base::fetch_add(unsigned long, std::memory_order) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/atomic_base.h:618

[ERROR] #1 0x7f6490a6120f in facebook::velox::io::IoStatistics::incRawBytesWritten(long) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/common/io/IoStatistics.cpp:58

[ERROR] #2 0x7f6492c0b0a3 in facebook::velox::dwio::common::FileSink::writeImpl(std::vector<facebook::velox::dwio::common::DataBuffer<char, void>, std::allocator<facebook::velox::dwio::common::DataBuffer<char, void> > >&, std::function<unsigned long (facebook::velox::dwio::common::DataBuffer<char, void> const&)> const&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/common/FileSink.cpp:82

[ERROR] #3 0x7f6492c0b907 in facebook::velox::dwio::common::LocalFileSink::write(std::vector<facebook::velox::dwio::common::DataBuffer<char, void>, std::allocator<facebook::velox::dwio::common::DataBuffer<char, void> > >&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/common/FileSink.cpp:168

[ERROR] #4 0x7f6492c0916e in facebook::velox::dwio::common::FileSink::writeWithLogging(std::vector<facebook::velox::dwio::common::DataBuffer<char, void>, std::allocator<facebook::velox::dwio::common::DataBuffer<char, void> > >&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/common/FileSink.cpp:60

[ERROR] #5 0x7f6492c0d76b in facebook::velox::dwio::common::FileSink::write(facebook::velox::dwio::common::DataBuffer<char, void>) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/common/FileSink.cpp:53

[ERROR] #6 0x7f6495d13111 in facebook::velox::text::BufferedWriterSink::flush() /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/text/writer/BufferedWriterSink.cpp:61 [ERROR] #7 0x7f6495d028de in facebook::velox::text::TextWriter::flush() /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/dwio/text/writer/TextWriter.cpp:110

[ERROR] #8 0x7f64a2f4a39e in facebook::velox::connector::print::PrintSink::appendData(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/connectors/print/PrintSink.cpp:100

[ERROR] #9 0x7f64a39d729d in facebook::velox::exec::TableWriter::addInput(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/exec/TableWriter.cpp:158

[ERROR] #10 0x7f6491bf0425 in facebook::velox::stateful::StatefulOperator::addInput(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:38

[ERROR] #11 0x7f6491bf395e in facebook::velox::stateful::StatefulOperator::pushOutput(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:76

[ERROR] #12 0x7f6491bf3ff5 in facebook::velox::stateful::StatefulOperator::getOutput() /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:61

[ERROR] #13 0x7f6491bf3a89 in facebook::velox::stateful::StatefulOperator::pushOutput(std::shared_ptrfacebook::velox::RowVector) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:77

[ERROR] #14 0x7f6491bf3ff5 in facebook::velox::stateful::StatefulOperator::getOutput() /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulOperator.cpp:61

[ERROR] #15 0x7f6491b8d6e9 in facebook::velox::stateful::StatefulTask::next(int&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/experimental/stateful/StatefulTask.cpp:116

[ERROR] #16 0x7f64a9cdf513 in velox4j::StatefulSerialTask::advance0(bool) /app/velox4j/src/main/cpp/main/velox4j/query/StatefulQueryExecutor.cc:136

[ERROR] #17 0x7f64a9cdfa0a in velox4j::StatefulSerialTask::advance() /app/velox4j/src/main/cpp/main/velox4j/query/StatefulQueryExecutor.cc:75

[ERROR] #18 0x7f64a9c96dca in upIteratorAdvance /app/velox4j/src/main/cpp/main/velox4j/jni/StaticJniWrapper.cc:77

[ERROR] #19 0x7f6ccccc5a6f ()

[ERROR] [ERROR] 0x6130004f1998 is located 24 bytes inside of 328-byte region [0x6130004f1980,0x6130004f1ac8)

[ERROR] freed by thread T143 (Source: MyTable) here:

[ERROR] #0 0x7f6ceb310d07 in operator delete(void*, unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cpp:172

[ERROR] #1 0x7f64a2f4fc41 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >::deallocate(std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2>*, unsigned long) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/ext/new_allocator.h:145

[ERROR] #2 0x7f64a2f4fc41 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > >::deallocate(std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >&, std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2>*, unsigned long) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/alloc_traits.h:496

[ERROR] #3 0x7f64a2f4fc41 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > >::~__allocated_ptr() /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/allocated_ptr.h:74

[ERROR] #4 0x7f64a2f4fc41 in std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2>::_M_destroy() /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr_base.h:538 [ERROR]

[ERROR] previously allocated by thread T143 (Source: MyTable) here:

[ERROR] #0 0x7f6ceb30fe87 in operator new(unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cpp:99

[ERROR] #1 0x7f64a2f4b5c7 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/ext/new_allocator.h:127

[ERROR] #2 0x7f64a2f4b5c7 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/alloc_traits.h:464

[ERROR] #3 0x7f64a2f4b5c7 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2> >&) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/allocated_ptr.h:98

[ERROR] #4 0x7f64a2f4b5c7 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics>(facebook::velox::io::IoStatistics*&, std::_Sp_alloc_shared_tag<std::allocatorfacebook::velox::io::IoStatistics >) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr_base.h:648

[ERROR] #5 0x7f64a2f4b5c7 in std::__shared_ptr<facebook::velox::io::IoStatistics, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocatorfacebook::velox::io::IoStatistics>(std::_Sp_alloc_shared_tag<std::allocatorfacebook::velox::io::IoStatistics >) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr_base.h:1342

[ERROR] #6 0x7f64a2f4b5c7 in std::shared_ptrfacebook::velox::io::IoStatistics::shared_ptr<std::allocatorfacebook::velox::io::IoStatistics>(std::_Sp_alloc_shared_tag<std::allocatorfacebook::velox::io::IoStatistics >) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr.h:409

[ERROR] #7 0x7f64a2f4b5c7 in std::shared_ptrfacebook::velox::io::IoStatistics std::allocate_shared<facebook::velox::io::IoStatistics, std::allocatorfacebook::velox::io::IoStatistics>(std::allocatorfacebook::velox::io::IoStatistics const&) /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr.h:863

[ERROR] #8 0x7f64a2f4b5c7 in std::shared_ptrfacebook::velox::io::IoStatistics std::make_sharedfacebook::velox::io::IoStatistics() /opt/rh/devtoolset-11/root/usr/include/c++/11.4.0/bits/shared_ptr.h:879 [ERROR] #9 0x7f64a2f4b5c7 in facebook::velox::connector::print::PrintSink::createWriter(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) /app/velox4j/src/main/cpp/build/_deps/velox-src/velox/connectors/print/PrintSink.cpp:44

[ERROR] [ERROR] Thread T146 (Legacy Source T) created by T143 (Source: MyTable) here:

[ERROR] #0 0x7f6ceb2b6676 in __interceptor_pthread_create ../../../../libsanitizer/asan/asan_interceptors.cpp:216

[ERROR] #1 0x7f6ce5cadb91 in os::create_thread(Thread*, os::ThreadType, unsigned long) (/usr/lib/jvm/java-11-openjdk-11.0.23.0.9-2.el7_9.x86_64/lib/server/libjvm.so+0xc67b91)

[ERROR] #2 0x7f6ce59d49ac in JVM_StartThread (/usr/lib/jvm/java-11-openjdk-11.0.23.0.9-2.el7_9.x86_64/lib/server/libjvm.so+0x98e9ac)

Copy link

@shuai-xu shuai-xu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the work.

@shuai-xu shuai-xu merged commit 04965e9 into bigo-sg:gluten-0530 Oct 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants